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Introduction 


With the assistance of the Lumina Foundation, the State Higher Education Executive 
Officers (SHEEO) staff has been working to develop a broad, up-to-date database of 
policy relevant information for the states and to create analytical studies to help 
state leaders identify priorities and practices for improving policies and 
performance across the higher education landscape. 

The proj ect is entitled State Policy Resource Connections (SPRC). Its stated 
objective is to provide SHEEOs with rapid, easy access to policy relevant data, 
and to create analytical studies to help state leaders identify priorities and 
develop strategies for improving policies and performance. 

The proj ect resulted in a data warehouse built through Microsoft SQL Server, which 
allowed for streamlined uploading, cleaning and organizing IPEDS data sets from 
eight institutional surveys (Completions, Enrollment, Employees, Fall Staffing, 
Finance, Financial Aid, Graduation Rates, and Institutional Characteristics) for the 
past ten years. Four analytical reports were prepared and distributed based on the 
direction framed by the SPRC Board of Advisors. Each report included a national 
overview (publicly released) and state- specific institutional profiles (released to the 
respective state’s SHEEO). 

This report summarizes the findings and recommendations from a formal 
evaluation of the states’ responses to the project and the analytical reports 
described below. 

Appendix A provides a brief summary of the findings of each SPRC report and a link 
to the full report. The following is a short description of the four reports that were 
the focus of this evaluation: 

• Degree Productivity and Cost Trends (August 2010) provides a national and 
cross-state overview and analysis of trends in degree production and 
completion rates, costs per degree and completion, and enrollment in public 
higher education. 

• Certificate Production and the Race toward Higher Education Attainment 

(December 2010) provides national, regional and cross-state overviews of 
recent trends in certificate production and the types of certificates being 
produced. 
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• Staffing Trends in Public Colleges and Universities: A National Analysis 
2001-2009 (May 2011) analyzes postsecondary staffing characteristics and 
changes overtime by state, sector, and institutional type. 

• Degree Production Trends by Program Area: A National Analysis 2001 - 
2009 (August 2011) examines degree productivity by specific program area 
usingthe Classification on Instructional Programs (CIP). 


Evaluation Process 

Individual telephone interviews were conducted with SHEEO offices in twenty-four 
states between August 29 and October 13, 2011. SHEEO staff selected the states and 
scheduled the interviews. The author of this report personally interviewed all 
twenty-four states. Two states initially selected were not interviewed due to time 
and scheduling constraints. The individuals interviewed in each of the 24 states are 
listed in Appendix B. 

The interviews were framed around seven questions designed to assess whether the 
individual state SHEEO agency found the reports useful for informing policy and 
practice in that state, whether and how the SHEEO project helped the state with 
data analysis and policy development, and what recommendations, if any, that state 
official had for future SHEEO reports and analyses. The text of each interview 
question is provided below in the section entitled "Summary of Feedback 
Received." 

Interviewees were assured of confidentiality in an effort to obtain candid and 
thorough responses to the questions posed. Each interview lasted approximately 30 
minutes. Several interviewees asked for and received the author’s contact 
information to provide follow up responses to the questions. However, as of this 
writing, no follow up responses have been received. 


Summary of Findings and Recommendations 

1. A significant majority of the states interviewed in the course of this 
evaluation found the reports useful and have used the comparative analyses 
to inform decision makers (institutional presidents, boards, governors, 
legislators) and to identify priorities and develop strategies for 
improvement. 

2. SHEEO’s effort to develop and distribute comparative analytical reports is a 
good strategy for focusing the attention of busy state higher education 
leaders on important policy issues. 
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3. The subject matter addressed in the four comparative analyses was generally 
considered timely and relevant to ongoing work in the states. A majority of 
those interviewed indicated great interest in the general topic of degree 
productivity and associated costs. Several states commended the staffing 
trends report because it provided data and analysis not collected by the state. 

4. For states without robust data systems and large staffs, the reports were 
welcomed as filling a gap in terms of data collected and analytical capability. 
Even in states with rich data and analytical capacity, the studies provided the 
state and national average comparisons that put that state’s situation and 
policy initiatives into context. 

5. There is strong interest in the majority of the states for SHEEO to continue to 
keep these analytical studies current and to undertake new topics of interest 
to the states. 

6. SHEEO is considered a credible source and should leverage its position to 
undertake original research (e.g. through state surveys or data agreements to 
access student record systems) to provide information and analyze data that 
is not easily accessible. 

7. Two states indicated they were unaware of the studies until they were 
contacted to schedule the evaluation interview. SHEEO should review how 
the studies are distributed and ascertain if greater awareness could be 
achieved through webinars, or other ways to achieve broader distribution to 
key SHEEO agency staff in the states. 

8. Two states offered the view that the SHEEO studies are duplicative of other 
resources, such as the Delta Cost Project, and suggested that SHEEO should 
find its own "niche” for future reports. 

9. Three states expressed concerns about the timeliness and the consistency of 
the underlying data, while a large number of those interviewed expressed 
great interest in further disaggregation of the data as presented in the 
various reports, such as comparing states of similar size or demographic 
characteristics. 

10. In undertaking future studies, SHEEO should consider how best to engage the 
states in the identification of topics of greatest interest or highest priority. 
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Summary of the Feedback Received 


Question 1. Is the information provided in the past studies useful for 
informing policy and practice in your state? If so, in what ways? If not, what 
types of information would be more helpful? 

A large majority of the states interviewed as part of this evaluation (17 out of 24) 
indicated that the reports were useful and relevant to policy and practice. Five 
states had a positive response to the studies but for a variety of reasons had not 
utilized them. Of these, three states indicated that while the reports appeared to be 
useful and topical, the particular circumstances in that state (e.g. leadership 
transition) had made it difficult for them to apply the findings to policy and practice 
in that state, and two states were simply not aware of the studies until they were 
contacted re questing they be part of the evaluation process. Only two states said 
the reports were not useful or were duplicative of other publicly available 
resources. 

With one exception, the responses from the state SHEEO were quite positive, even 
enthusiastic, about the value and usefulness of the four studies. In the states 
without a SHEEO or in several states when the state SHEEO was unavailable for the 
interview, the SHEEO staff or institutional representative interviewed tended to 
focus more on their technical concerns about the underlying data and the 
methodology for making national comparisons. It is fair to conclude that the 
primary audience for this SHEEO project - the state SHEEOs themselves - were 
extremely positive about the usefulness of the studies and strongly supportive of 
SHEEO continuing this work. 

Among those states with a positive response, many volunteered that the 
comparative analyses of degree productivity and cost trends provided an important 
national context for ongoing work in the respective states. Several indicated that 
the report on certificate production had prompted a reexamination of their 
performance funding formulas. Several other states commented that the report on 
staffing trends had been of particular interest and was especially useful because it 
had not been addressed by other research organizations. 

Even among the states with a strong positive response to the analytical studies, 
there was considerable discussion during the interview of technical concerns about 
the quality, timeliness and consistency of the data underlying the reports. Two 
states raised questions about the validity of the findings claiming that their previous 
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feedback (provided during an earlier review process) had not been incorporated 
into the final reports. 

In general, a significant majority of the states said the reports were helpful in that 
they focused attention on issues that were timely and relevant, provided a depth of 
analysis that is especially needed as states cut back on staffing and research 
capabilities, and even for those states with robust data and analytical capability, the 
national and state-by-state comparisons provided context that was otherwise 
missing in that state’s own work. 

Question 2. What recommendations do you have for future reports in these 
areas or other areas? 

This question generated a large and diverse set of suggestions that is difficult to 
summarize. The most commonly mentioned topics for future research or reports 
were: 

• Linking data on educational outcomes and degree productivity with 
workforce and employment needs; 

• Continuing SHEEO’s current focus on degree productivity and cost trends; 
keeping the data current; focusing the analysis on smaller groupings of states 
in regions or by grouping states of similar size and population 
characteristics; 

• Disaggregation of degree productivity cost trends and completion metrics by 
sector and institutional mission; 

• Completion statistics by states, disaggregated by student characteristics 
(race, gender, income); 

• More information on certificate production, data on completions for 
certificates that are industry recognized 

• Financial aid and student indebtedness; 

• A comparative analysis of graduate degree production across the states; 

• A compendium of best practices and innovations across the states on a 
variety of topics (completion and attainment; financial performance and 
outcomes funding; staffing and workload issues); 

• Good metrics for accountability measures commonly used (graduation rates, 
degree production, expenditures) 

• An analysis of the cost of "failure" - what is the lost investment or lost 
opportunity associated with not achieving national attainment goals? 

This list barely scratches the surface of the rich response received to the question of 
future SHEEO analytical efforts. The variety and number of suggestions is testament 
to both the hunger for better information and the credibility of SHEEO in filling that 

gap- 
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Question 3. Do these studies fill a gap in terms of the types of data collected in 
your state? 

Question 4. Do these studies fill a gap in terms of your state’s capacity to 
analyze data of these and other issues? 

These questions are closely related and states tended to respond to both similarly 
depending on the depth and breadth of state data collection and research capability. 
A majority of the states (19 of 24) responded that the SHEEO reports either filled a 
data and analytical capacity gap or complemented the work the state was doing. 
Having national and state-by state comparisons was seen as especially helpful even 
in states with robust data systems. The SHEEO reports helped states to see national 
trends, which provided perspective and focused the state on areas of concern. One 
state commented that if they knew in advance what research SHEEO was 
undertaking, it would allow them to divert their own resources to other pressing 
needs. Another state suggested that as states improve their data capacity, it would 
be helpful to have role clarification. What should SHEEO focus on and where should 
the states direct their attention? Several states indicated that while they collected a 
lot of data, the SHEEO report on staffing trends filled a particular gap. 

Five states responded that the reports did not fill either a data gap or an analytical 
capacity gap. One state expressed concern that the reports provided institutional 
comparisons, which was not the policy approach preferred in that state. Rather, 
SHEEO should have focused on sectors or separated the analysis by institutional 
mission. Another state commented that the SHEEO reports caused extra work 
because the state had to "reconcile and contextualize the message." This comment 
was offered by an individual from an institutional research office, whose job is to 
review and report the findings of a large number of national reports. 

Question 5. In addition to the reports for which we've asked your feedback, 
what other data sources have you found helpful? How and why are these 
sources useful? What are the issues that make them more or less useful? 

All of the states rely on a variety of data sources. Commonly mentioned are IPEDS, 
NCHEMS, SREB, WICHE and the Delta Cost Project. States also rely on the work of 
Complete College America (CCA), the College Board for SAT reports, the Census, the 
National Center for Education Statistics (NCES) and the National Student 
Clearinghouse (NCS). Several states whose responsibilities include community 
colleges mentioned Achieving the Dream as a source of metrics and a model for 
collaboration. SHEEO’s State Higher Education Finance (SHEF) annual reports were 
also mentioned. 

When asked how and why these sources are useful, the response focused on 
particular needs met by the data source mentioned. For example, the CCA was 
praised for collecting attainment and degree completion data from the states. 
NCHEMS was mentioned for a variety of products and Patrick Kelley’s work was 
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highlighted. Two states expressed the view that the SHEEO reports were 
duplicative of the work undertaken by the Delta Cost Project. Several expressed the 
hope that SHEEO might become a "one-stop shop" providing easy access to these 
commonly used data sources. 

Question 6. Would you find it useful to discuss these findings with your 
colleagues in other states? And if so, how? 

All but one state responded positively to this question. States are interested in 
discussing the issues raised by these reports with their colleagues at future SHEEO 
meetings, policy conferences, and through other means such as webinars. One 
suggested that the next step should be a SHEEO sponsored discussion about the 
results of this initiative (SPRC), what has been accomplished to date and what 
should be undertaken next. A number of states indicated that the conversation 
should be structured or focused among regions or by highlighting best practices 
among states with the best outcomes on various measures. Several indicated that 
this work should be augmented by a process like Achieving the Dream where 
colleagues meet face to face to share best practices and develop consensus on the 
metrics. 

Question 7. What are the advantages and disadvantages of presenting data to 
you in different formats? 

a. Written comparative reports like the SPRC reports, 

b. Web-based resources which you can use for your own data 
manipulation, 

c. Other formats (please describe). 

Most states indicated a preference for both written analytical reports and web- 
based resources they can access for data manipulation. Two states said that access 
to data was the preferred format. Two states preferred the written analytical 
reports and indicated that by "written reports" they meant PDF files that they could 
easily transmit to others. Several states shared the view that the comparative 
analytical reports raised issues and pointed to trends that were difficult for them to 
raise themselves but that it was important to access the data to "slice and dice” it to 
address that state’s context. It was suggested by several that the reports could be 
improved by providing clearer statements, especially on the charts and graphs, of 
the "take-aways." As one said, "you need a teaser and then the ability to go to the 
website for the data." 


Additional Comments 

At the conclusion of the formal questions, states were asked if they had any 
additional comments they would like to share. Many thanked SHEEO for 
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undertaking this initiative and indicated that SHEEO is a credible source of 
information that they rely on. A few additional comments are highlighted: 

• It would be good to have an advisory board of SHEEO members with 
expertise in IPEDS. 

• SHEEO is one of the best at helping us confront difficult issues and set the 
stage for a safer conversation. SHEEO brings credibility to these 
conversations. 

• We recommend that SHEEO develop this body of work into an ongoing 
rigorous set of reports. 

• The SHEEO work gives us "pillars and platforms" to do our public policy 
work. 

• I like that SHEEO is attuned to users and works with the states in the design 
of these studies. 

• The information is useful and we hope it keeps coming! All of this work has 
dovetailed nicely with the work of the National Governors Association and 
Complete College America and the national agenda on completions. 

• SHEEO is in a good position to continue its work to provide national and 
state comparisons. 

• The biggest help SHEEO could give the states would be to help us understand 
the WHY of the national trends, not just the WHAT. 

• SHEEO has great credibility. So far these reports are right on target. 

• It would be really cool if SHEEO would provide a centralized place to access 
IPEDS and other data sources, focused around research topics. 

• The more we can move toward interpretation of the data, the better. SHEEO 
could help with interpretation and create appropriate benchmarks. 

• This is a very worthwhile activity. Fine-tuning is always appropriate, but 
let’s do more! 


Conclusions 

SHEEO’s State Policy Resource Connections (SPRC) initiative has been well received 
by its intended users and there is strong interest among the states interviewed in 
assisting SHEEO in the development of next steps on this initiative and identifying 
priority areas of research. When asked for suggestions for future research, the 
great variety and number of recommendations received is testament to both the 
hunger for better information and the credibility of SHEEO in filling that gap. 
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Appendix A: Summary of SPRC Reports 


Degree Productivity and Cost Trends 
August 2010 

http://www.sheeo.org/pubs/Degree Production and Cost Trends.pdf . 

This report provides a national and cross-state overview and analysis of trends in 
degree and completion production, costs per degree and completion, and 
enrollment in public higher education. Specific findings included: 

• Changes in FTE enrollment and awards (degrees and completions) are 
positive in the majority of states. 

• In all Carnegie groupings, degree and completion productivity exceed FTE 
enrollment growth. 

• Over the past decade, costs per degree and completion have been reasonably 
stable in every Carnegie grouping, with a modest amount of variation among 
the groupings. The cost per degree grew by 6 percent nationally in the first 
five years, and it declined by 6 percent in the last five years. 

Certificate Production and the Race toward Higher Education Attainment 
December 2010 

http://www.sheeo.org/pubs/pcn/Uploads /Final%20 Report.pdf . 

This report provides a national, regional and cross-state overview of recent trends 
in certificate production and the types of certificates driving production. Specific 
findings included: 

• Nationally, certificate and associate degree production grew over the last ten 
years, primarily fueled by strong growth in the first part of the decade, and 
with production in the public sector occurring at associates institutions. 

• Certificates less than one year and certificates equal to one year but less than 
two years represent the majority of certificates conferred by associates 
institutions. 

• Regionally, the South experienced the strongest growth in FTE, certificates, 
and associates degrees. 

• Ohio, Kentucky, Arkansas, N evada, and New Mexico saw the largest growth 
margin in certificates awarded. 
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Staffing Trends in Public Colleges and Universities: A National Analysis 2001 -2009 
May 2011 

http://www.sheeo.org/pubs/Fall Staffing-Final05-24-2011.pdf . 

This report analyzes postsecondary staffing characteristics and changes over time 
by state, sector, and institution type. Specific findings included: 

• Between 200 1 and 2009, America’s public colleges and universities 
experienced a decline in total staff per 100 student FTE. Although both 
student FTE and total staff have increased, staffing levels have increased at a 
slower rate. 

• Part-time instructional staff (Faculty and Graduate Assistants) per 100 
student FTE has stayed relatively constant while full-time instructional staff 
per 100 student FTE has declined. Public colleges and universities may be 
meeting increasing student demand with part-time staff rather than creating 
full-time positions. 

• Staffing areas such as Clerical and Secretarial, Executive/Administrative and 
Management, and Maintenance and Skilled Crafts were more likely to see 
declines in staff per 100 student FTE while areas that are directly involved in 
serving students such as Faculty, Graduate Assistants, and Other Professional 
were likely to see increases in staff per 100 student FTE but with variations. 
This suggests that from 2001 to 2009 the sector has achieved some 
economies of scale, particularly in operational support. 

• Carnegie groups with more emphasis on research employ more Other 
Professional staff per 100 student FTE, and Other Professionals make up a 
larger portion of their overall staff. 

Degree Production Trends by Program Area: A National Analysis 2004-2009 
August 2011 

http://www.sheeo.org/pubs/Degree Production BY Program Area.pdf . 

This report provides an analysis that builds upon the previous Degree Productivity 
and Cost Trends report by examining degree productivity by specific program area 
using the Classification of Instructional Programs (CIP). Specific findings included: 

• Between 2004 and 2009, student FTE increased by 11 percent, while the 
number of degrees awarded grew by 13 percent and total completions grew 
by 15 percent. The total number of certificates awarded increased 
significantly (26 percent) over the same five year time period. 
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Across disciplines, Health (37%) experienced the largest increase in the 
number of total completions, while total completions in T rades grew by 24 
percent. Education (2%) and STEM (4%) experienced the lowest growth in 
total completions. 

Across the seven program areas: Associates institutions produced the 
highest number of completions with Arts and Humanities (54%), Health 
(65%) and Trades (83%); Baccalaureate, Masters and Doctoral institutions 
produced the highest number of completions within Business and 
Communications (35%), Education (54%) and Social and Behavior 
Sciences/Human Services (34%); Research - Very High Activity Institutions 
produced the highest number of total completions in STEM (37%). 

With regards to market share, the majority of disciplines maintained their 
market share levels from 2004-2009, with the exception of Health and STEM 
fields. 
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Appendix B: List of Participants 


Alabama 

Gregory Fitch, Executive Director 
Alabama Commission on Higher Education 

Connecticut 

Mike Meotti, Executive Vice President 
Connecticut Board of Regents for Higher Education 

Florida 

Frank Brogan, Chancellor 

Tim Jones, Chief Financial Officer 

Florida Board of Governors, State University System 

Idaho 

Mike Rush, Executive Director 
Leah Schultz, Research Director 
Idaho Board of Education 

Illinois 

George W. Reid, Executive Director 
Illinois Board of Higher Education 

Indiana 

Teresa Lubbers, Commissioner 

Ken Sawyer and Molly Chamberlain, staff 

Indiana Commission for Higher Education 

Iowa 

Diana Gonzalez, Chief Academic Officer 
Iowa Board of Regents 

Kansas 

Andy Tompkins, President and CEO 
Diane Duffy, Vice President for Finance 
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Gary Alexander, Vice President for Academic Affairs 
Kansas Board of Regents 

Kentucky 

Bob King, President 

John Hayek, Senior Vice President for Budget 
Kentucky Council on Postsecondary Education 
Louisiana 

James Purcell, Commissioner of Higher Education 
Board of Regents 

Massachusetts 

Richard Freeland, Commissioner 

Alison MacDonald, Director of Finance and Administration 
Jonathan Keller, Associate Commissioner for Planning and Research 
Department of Higher Education 

Missouri 

Timothy Wittmann, Research Associate 
Missouri Department of Higher Education 

Nebraska 

Marshall Hill, Executive Director 

Nebraska’s Coordinating Commission for Postsecondary Education 
New lersev 

Betsy Garlotti, Director of Finance and Research 

Alan Gunther, Senior Policy Advisor 

New Jersey Commission on Higher Education 

Oklahoma 

Glen Johnson, Chancellor 

Houston Davis, Vice Chancellor 

Oklahoma State Regents for Higher Education 

Oregon 

Robert Kieran, Assistant Vice Chancellor 
Oregon University System 

Pennsylvania 

John Cavanaugh, chancellor 

Pennsylvania State System of Higher Education 

South Dakota 

Jack Warner, Executive Director and Chief Executive Officer 
South Dakota Board of Regents 
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Tennessee 
David Wright 

Associate Executive Director 
Tennessee Higher Education Commission 


Texas 

Raymund Paredes, Commissioner 

Texas Higher Education Coordinating Board 

Virginia 

Peter Blake, Interim Director 

Virginia State Council of Higher Education 

West Virginia 

Brian Noland, Chancellor 

West Virginia Higher Education Policy Commission 

Wisconsin 
Heather Kim 
Sue Buth 

University of Wisconsin Office of Policy Analysis and Research 

Wyoming 
Don D. Richards 

Director of Government and Community Affairs 
University of Wyoming 
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